Method in which camera module transfers image data and computer therefor

ABSTRACT

A camera system that transfers static image data while reducing power consumption is provided. A static image application issues a frame transfer command at a certain transfer period to a camera module. In each transfer period, the camera module wakes up from a suspend state and generates static image data. After the end of the transfer, the camera module transitions to the suspend state. The camera module is able to transition to the suspend state at each transfer period and thus is able to reduce power consumption.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a national patent application and claims priority toJapanese Patent Application Number 2011-196887 entitled “METHOD IN WHICHCAMERA MODULE TRANSFERS IMAGE DATA AND COMPUTER THEREFOR” and filed on 9Sep. 2011 for Susumu Shimotono, Jun Sugiyama, and Hideki Kashiyama,which is incorporated herein by reference.

FIELD

The present invention relates to a technique in which a camera modulecontinuously transfers image data to a host device at a frame rate lowerthan that of dynamic image data, and more particularly to a techniquefor reducing power consumption of the camera module and a host deviceduring data transfer.

BACKGROUND

A singly-existing digital camera can be roughly classified into a stillcamera which photographs a static image and a video camera thatphotographs a dynamic image. In both cases of the still camera and thevideo camera, the camera displays an image of a subject on aliquid-crystal display monitor when the camera is powered on and entersa photographing preparation state. A user takes a static image or adynamic image while checking the image on the liquid-crystal displaymonitor and records data into a camera body.

In some cases, a video phone or a camera module intended to photographan ambient environment may be mounted into a host device such as anotebook personal computer (hereinafter, referred to as “note PC”), atablet computer, or a feature phone. Generally, image data photographedby a camera module is received by an application of a note PC and thenrecorded into a recording medium of the host device. Therefore, it canbe said that the camera module built in the note PC is combined with theapplication that processes and records image data, thereby constructinga camera system compatible with a singly-existing digital camera.

In this kind of camera system, it is also possible to implement anapplication program (application) that requires only static images. Thecamera module transfers image data to the system at a dynamic-imageframe rate of 30 fps after entering the photographing preparation state.The transfer of dynamic image data in this case aims at monitoringimages to be photographed, and therefore generally the number of pixelsis set lower than that of the originally intended dynamic image data.The term “frame rate” means the number of frames (fps) generated ortransferred per unit time on the assumption that one frame is image datacomposed of pixel signals output from an aggregation of a plurality ofimage pickup devices for generating one static image. One static imagecorresponds to one frame.

The application determines the timing for selecting a static image outof the received dynamic image data for monitor display. Then, the cameramodule transfers static image data having the original number of pixelsat the determined timing. Alternatively, in the middle of transferringthe dynamic image data having the original number of pixels, the staticimage data can be directly acquired from there. More specifically, inorder to acquire a static image by a camera system, it is necessary toacquire dynamic image data having the number of pixels for monitordisplay or the original number of pixels for a predetermined time periodduring which an application or a user selects a required static image.

Patent Document 1 provides emergency supervisory equipment that achievesa reduction in power consumed to process a dynamic image. The emergencysupervisory equipment normally pauses the operation of a dynamic imageprocessed part and starts the operation of the dynamic image processedpart when triggered by voice anomaly detection. Patent Document 2discloses an image display system that reduces the power consumption ofthe side that outputs static image data when storing the static imagedata in a dynamic image format and then outputting it to an imagedisplay device. A digital video camera performs a transfer operation ofstatic image data based on an image transfer request command from adigital television and thereafter operates in a low power consumptionmode until receiving an image transfer request command sent from thedigital television.

Non-patent Document 1 prescribes Still Image Capture for acquiring astill image from a camera module connected by USB. This documentdescribes that a host device once halts the video streaming and thenacquires a still image and thereafter returns to a video streamingtransfer mode or transfers a still image by using a dedicated bulk stillimage pipe without halting video streaming.

Non-patent Documents 2 and 3 describe selective suspend of a USB devicereferred to as “low power mode.” In the selective suspend, a USB clientdriver sends a request packet to a USB bus driver when it is determinedthat the USB device is in an idle state. When all USB devices connectedto the USB hub enter the idle state, thereupon the USB bus driver makesthe USB bus connecting to the USB hub transition to the idle state.Then, the USB device that has detected the state transition of the bustransitions to a selective suspend state.

Patent Document 1—Japanese Patent Application Laid-Open No. 2004-120595

Patent Document 2—Japanese Patent Application Laid-Open No. 2008-187536

Non-patent Document 1—USB Device Class Definition for Video Devices, REvision 1.1, Jun. 1, 2005

Non-patent Document 2—Universal Serial Bus Specification Revision 2.0

Non-patent Document 3—Power saving of using USB Selective SuspendSupport Whitepaper, Intel Mobile Platforms Group Version 0.3, May 20,2003, Kris Fleming

SUMMARY

The camera module is able to transition to a suspend state unless ittransfers image data. In conventional camera systems, however, it isnecessary to send dynamic image data for monitor display even if thehost side needs only static image data. Therefore, even in the casewhere static image data is to be transferred at a regular interval, thecamera module cannot transition to the suspend state until the datatransfer of the dynamic image ends. Further, data traffic increasesbeyond necessity because dynamic image data is transferred in aphotographing preparation stage. Moreover, the CPU load for processingstreaming data increases, by which the power consumption of the hostdevice also increases more than necessary.

A note PC contains various applications using dynamic image data such asthose for a video phone and videography. When the note PC operates onbattery-power, the power consumption of the camera system needs to beminimized. Therefore, when the camera system is operated, generally anAC/DC adapter is connected to the camera system to use AC power.Recently, however, there are applications being developed that want toacquire static image data continuously at a frame rate lower than theframe rate of dynamic image data.

For example, there could exist an application that determines thedirection of the face of a user using a note PC in a meeting byacquiring a static image of the user's face every one or two seconds.When the face of the user points in a direction different from thedirection toward the screen, or the user's eyes are turned away from thescreen, the application would stop the backlight of a display ordecrease the clock frequency of a processor to shift the note PC to apower-saving state. The note PC would then return to the normal statewhen the user's face points in the direction of the screen.

A conventional camera system cannot be used when this kind ofapplication is put into practice on a note PC operating on battery powerbecause of the high power consumption. Moreover, because it is difficultto use dynamic images when a note PC is operating on battery-power in aremote location, it may be required in some cases, to use static imagessent at certain intervals in order to hold a teleconference.

Therefore, the object of the present invention is to provide a method oftransferring image data from a camera module to a host device whilereducing power consumption. Another object of the present invention isto provide a method of continuously transferring static image data at aframe rate lower than the frame rate of dynamic image data whilereducing the power consumption of a camera system. Still another objectof the present invention is to provide a method of transferring imagedata while satisfying both a maintained frame rate requested by the hostdevice and a reduction in power consumption. Further, another object ofthe present invention is to provide a camera system, a camera module, acomputer, and a computer program for implementing the methods.

The present invention provides a method of transferring image data froma camera module, which includes an image sensor, to a host device. Thecamera module resides in a low-power state, which requires less powerconsumption than in a photographing state, during a period of time otherthan a period of time necessary for transferring the image data. Thehost device issues ordered transfer requests of image data to the cameramodule. Upon receiving the transfer request, the camera moduletransitions from the low-power state to the photographing state. Thecamera module, which has transitioned to the photographing state,generates image data and transfers the image data to the host device. Inresponse to the end of the transfer, the camera module transitions tothe low-power state.

According to the present invention, the camera module does not need totransfer dynamic image data for monitor display and the host devicetransfers only required image data. Therefore, unless a transferoperation is performed, the camera module is able to transition to thelow-power state. Thus, not only the power consumption of the cameramodule, but also the traffic of the image data is reduced to therequired minimum, thereby also enabling a reduction in the powerconsumption of the bus and the CPU that process the traffic.

The transfer request may be issued at a certain transfer period. Thehost device is able to set the lower limit of an acceptable frame rateor the upper limit of a transfer period. In the static image transfermode, the camera module needs to transition to the low-power statewithout fail in each transfer period. Therefore, when a practicaltransfer period is set between the minimum value of the transfer periodcalculated from an actual operating time in the photographing state ineach transfer period and the maximum value of the transfer periodcalculated from the frame rate requested by the host device, the timeduring which the camera module resides in the low-power state can besecured while satisfying the request from the host device. Although theimage data transferred in each transfer period may be composed of aplurality of frames, one frame of image data enables an increase in timeduring which the camera module resides in the low-power state, therebyreducing the power consumption to the maximum.

While the actual operating time depends on the inherent performance ofthe camera module, the actual operating time can be reduced by settingthe parameter values, which have been previously acquired withoutcalibration in each transfer period, into the camera module. A reductionin the actual operating time enables an increase in time during whichthe camera module resides in the low-power state in each transfer periodor a decrease in the transfer period relative to the time during whichthe camera module resides in the low-power state (an increase in theframe rate). This achieves both a reduction in power consumption of thecamera module and an increase in the frame rate.

The previously acquired parameter values can be those acquired byrunning the calibration so as to adapt the camera module tophotographing conditions in a first transfer period. The method offixing the parameter values in the second and subsequent transferperiods by skipping the calibration is based on the assumption that thephotographing conditions at the beginning of photographing do notchange.

In actuality, however, the host device cannot acquire image data havinga desirable image quality in some cases because the photographingconditions change from those at the beginning of photographing. In orderto cope with the changing conditions, the host device is able to requestthe camera module to rerun the calibration on the basis of the result ofevaluating static image data. The host device or the camera module isable to set parameter values acquired by rerunning the calibration. Theparameter may include rolling shutter exposure time and white balance.

The host device is able to change the interval between transfer requestsby evaluating regularly received image data. For example, the intervalis shortened in the case where a subject frequently changes and theinterval is prolonged in the case where the subject infrequentlychanges. The change in the interval between transfer requests enablesthe transfer of unnecessary image data to be omitted or the image datato be transferred at required timing, by which the power consumption ofthe camera system can be effectively reduced while sufficientlysatisfying the requirement of the host device.

The host device may be a computer. In this case, the camera module maybe connected to the system via a USB interface. The present invention isable to transfer static image data at a frame rate of, for example, oneto two fps while remarkably reducing the power consumption of the camerasystem. This enables implementation of an application that uses imagedata transferred at such a frame rate in a battery-powered portablecomputer.

According to the present invention, there can be provided a method oftransferring image data from a camera module to a host device whilereducing power consumption. Furthermore, according to the presentinvention, there can be provided a method of transferring static imagedata continuously at a frame rate lower than the frame rate of dynamicimage data while reducing power consumption of a camera system. Stillfurther, according to the present invention, there can be provided amethod of transferring image data while satisfying both a maintainedframe rate requested by the host device and a reduction in powerconsumption. Furthermore, according to the present invention, there canbe provided a camera system, a camera module, a computer, and a computerprogram for implementing the above methods.

BRIEF DESCRIPTION OF THE DRAWINGS

In order that the advantages of the invention will be readilyunderstood, a more particular description of the invention brieflydescribed above will be rendered by reference to specific embodimentsthat are illustrated in the appended drawings. Understanding that thesedrawings depict only typical embodiments of the invention and are nottherefore to be considered to be limiting of its scope, the inventionwill be described and explained with additional specificity and detailthrough the use of the accompanying drawings, in which:

FIG. 1 is a functional block diagram illustrating the hardwareconfiguration of a camera system implemented by a note PC;

FIG. 2 is a functional block diagram illustrating a schematicconfiguration of a camera module;

FIG. 3 is a functional block diagram illustrating the configuration ofsoftware loaded into the note PC constructing the camera system;

FIG. 4A is a diagram describing the operating state of the camera modulein a static image transfer mode;

FIG. 4B is another diagram describing the operating state of the cameramodule in a static image transfer mode;

FIG. 4C is another diagram describing the operating state of the cameramodule in a static image transfer mode;

FIG. 5 is a flowchart describing a procedure for transferring image datain the static image transfer mode; and

FIG. 6 is a diagram describing the feature of the static image transfermode.

DETAILED DESCRIPTION

Reference throughout this specification to “one embodiment,” “anembodiment,” or similar language means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment of the present invention. Thus,appearances of the phrases “in one embodiment,” “in an embodiment,” andsimilar language throughout this specification may, but do notnecessarily, all refer to the same embodiment.

Furthermore, the described features, structures, or characteristics ofthe invention may be combined in any suitable manner in one or moreembodiments. In the following description, numerous specific details areprovided, such as examples of programming, software modules, userselections, network transactions, database queries, database structures,hardware modules, hardware circuits, hardware chips, etc., to provide athorough understanding of embodiments of the invention. One skilled inthe relevant art will recognize, however, that the invention may bepracticed without one or more of the specific details, or with othermethods, components, materials, and so forth. In other instances,well-known structures, materials, or operations are not shown ordescribed in detail to avoid obscuring aspects of the invention.

The schematic flow chart diagrams included herein are generally setforth as logical flow chart diagrams. As such, the depicted order andlabeled steps are indicative of one embodiment of the presented method.Other steps and methods may be conceived that are equivalent infunction, logic, or effect to one or more steps, or portions thereof, ofthe illustrated method. Additionally, the format and symbols employedare provided to explain the logical steps of the method and areunderstood not to limit the scope of the method. Although various arrowtypes and line types may be employed in the flow chart diagrams, theyare understood not to limit the scope of the corresponding method.Indeed, some arrows or other connectors may be used to indicate only thelogical flow of the method. For instance, an arrow may indicate awaiting or monitoring period of unspecified duration between enumeratedsteps of the depicted method. Additionally, the order in which aparticular method occurs may or may not strictly adhere to the order ofthe corresponding steps shown.

[Hardware Configuration of Camera System]

FIG. 1 is a functional block diagram illustrating the hardwareconfiguration of a camera system implemented by a note PC 10. The CPU 11includes a memory controller and a PCI Express controller and isconnected to a main memory 13, a video card 15, and a chip set 19. AnLCD 17 is connected to the video card 15. The chip set 19 includes areal time clock (RTC) and SATA, USB, PCI Express, and LPC controllers. AHDD 21 is connected to the SATA controller. A USB controller is composedof a plurality of hubs constructing a USB host controller, a route hub,and an I/O port.

A camera module 100 is a USB device compatible with the USB 2.0 standardor the USB 3.0 standard. The camera module 100 is connected to the USBport of the USB controller via one or three pairs of USB buses 50, whichtransfer data using a differential signal. The USB port, to which thecamera module 100 is connected, may share a hub with another USB device.Preferably the USB port is connected to a dedicated hub of the cameramodule 100 in order to effectively control the power of the cameramodule 100 by using a selective suspend mechanism of the USB system. Thecamera module 100 may be of an incorporation type in which it isincorporated into the housing of the note PC 10 or may be of an externaltype in which it is connected to a USB connector attached to the housingof the note PC 10.

Moreover, the camera module 100 may be connected to the note PC 10 via awireless USB connection. The camera system according to the presentinvention is able to transfer image data in a new static image transfermode in addition to a dynamic image transfer mode that hasconventionally existed. The camera module 100 transfers image data ineither of the dynamic image transfer mode and the static image transfermode.

Here, the system of the note PC 10 constructing the camera systemincludes hardware such as the CPU 11, the chip set 19, and the mainmemory 13. The system of the note PC 10 also includes software such as astatic image application 202, device drivers 207, 209, 211, and 231 ofthe respective layers, a static image transfer service 205, and anoperating system 203 illustrated in FIG. 3. The static image transfermode is used by the static image application 202 to acquire a staticimage. The contents of the static image transfer mode will be describedlater.

An EC 25 is a microcontroller that controls the temperature of theinside of the housing of the computer 10 or controls the operation of akeyboard or a mouse. The EC 25 operates independently of the CPU 11. TheEC 25 is connected to a battery pack 27 and a DC-DC converter 29. The EC25 is further connected to a keyboard, a mouse, a battery charger, anexhaust fan, and the like. The EC 25 is capable of communicating withthe battery pack 27, the chip set 19, and the CPU 11. The battery pack27 supplies the DC-DC converter 29 with power when an AC/DC adapter (notshown) is not connected to the battery pack 27. The DC-DC converter 29supplies the device constructing the computer 10 with power.

[Camera Module]

FIG. 2 is a functional block diagram illustrating a schematicconfiguration of the camera module 100. The camera module 100 is able totransfer VGA (640×480), QVGA (320×240), WVGA (800×480), WQVGA (400×240),and other image data in the static image transfer mode. An opticalmechanism 101 includes an optical lens and an optical filter andprovides an image of a subject on an image sensor 103.

The image sensor 103 includes a CMOS image sensor that converts electriccharges, which correspond to the amount of light accumulated in photodiodes forming pixels, to electric signals and outputs the electricsignals. The image sensor 103 further includes a CDS circuit thatsuppresses noise, an AGC circuit that adjusts gain, an AD convertercircuit that converts an analog signal to a digital signal, and thelike. The image sensor 103 outputs digital signals corresponding to theimage of the subject. The image sensor 103 is able to generate imagedata at a frame rate of 30 fps when operating in the dynamic imagetransfer mode.

The CMOS image sensor is provided with an electronic shutter referred toas a “rolling shutter.” The rolling shutter controls exposure time so asto be optimal for a photographing environment with one or several linesas one block. In one frame period, or in the case of an interlace scan,the rolling shutter resets signal charges that have accumulated in thephoto diodes, and which form the pixels during one field period, in themiddle of photographing to control the time period during which light isaccumulated corresponding to shutter speed. In the image sensor 103, aCCD image sensor may be used, instead of the CMOS image sensor.

An image signal processor (ISP) 105 is an image signal processingcircuit which performs correction processing for correcting pixeldefects and shading, white balance processing for correcting spectralcharacteristics of the image sensor 103 in tune with the humanluminosity factor, interpolation processing for outputting general RGBdata on the basis of signals in an RGB Bayer array, color correctionprocessing for bringing the spectral characteristics of a color filterof the image sensor 103 close to ideal characteristics, and the like.The ISP 105 further performs contour correction processing forincreasing the resolution feeling of a subject, gamma processing forcorrecting nonlinear input-output characteristics of the LCD 17, and thelike.

An encoder 107 compresses image data received from the ISP 105. Anendpoint buffer 109 forms a plurality of pipes for transferring USB databy temporarily storing data to be transferred bidirectionally to or fromthe system. A serial interface engine (SIE) 111 packetizes the imagedata received from the endpoint buffer 109 so as to be compatible withthe USB standard and sends the packet to a transceiver 113 or analyzesthe packet received from the transceiver 113 and sends a payload to anMPU 115. When the USB bus 50 is in the idle state for a predeterminedperiod of time or longer, the SIE 111 interrupts the MPU 115 in order totransition to a suspend state. The SIE 111 activates the suspended MPU115 when the USB bus 50 has resumed.

The transceiver 113 includes a transmitting transceiver and a receivingtransceiver for USB communication. The MPU 115 runs enumeration for USBtransfer and controls the operation of the camera module 100 in order toperform photographing and to transfer image data. The camera module 100conforms to power management prescribed in the USB standard. When beinginterrupted by the SIE 111, the MPU 115 halts the internal clock andthen makes the camera module 100 transition to the suspend state as wellas itself.

When the USB bus 50 has resumed, the MPU 115 returns the camera module100 to the power-on state or the photographing state. The MPU 115interprets the command received from the system and controls theoperations of the respective units so as to transfer the image data inthe dynamic image transfer mode or the static image transfer mode. Whenstarting the transfer of the image data in the static image transfermode, the MPU 115 first performs the calibration of rolling shutterexposure time (exposure amount), white balance, and the gain of the AGCcircuit and then acquires optimal parameter values for the photographingenvironment at the time, before setting the parameter values topredetermined registers for the image sensor 103 and the ISP 105.

The MPU 115 performs the calibration of exposure time by calculating theaverage value of luminance signals in a photometric selection area onthe basis of output signals of the CMOS image sensor and adjusting theparameter values so that the calculated luminance signal coincides witha target level. The MPU 115 also adjusts the gain of the AGC circuitwhen calibrating the exposure time. The MPU 115 performs the calibrationof white balance by adjusting the balance of an RGB signal relative to awhite subject that changes according to the color temperature of thesubject.

When the image data is transferred in the dynamic image transfer mode,the camera module does not transition to the suspend state during atransfer period. Therefore, the parameter values once set to registersdo not disappear. In addition, when transferring the image data in thedynamic image transfer mode, the MPU 115 appropriately performscalibration even during photographing to update the parameter values ofthe image data.

In contrast, when image data is transferred in the static image transfermode, the camera module transitions to the suspend state for eachtransfer period. Therefore, depending on the connected image sensor, apart or all of the parameter values set to internal registers of theimage sensor may disappear because it is not assumed that the cameramodule gradually transitions to the suspend state while intermittentlyrepeating photographing in use of the camera module. Therefore, there isno design incentive for holding the values of the internal registers inthe suspend state. In the static image transfer mode, calibration isperformed only in photographing during the first transfer period. Inphotographing during the subsequent transfer periods, the parametervalues acquired by the first calibration can be set for each wakeup.

Skipping the calibration in each of the second and subsequent transferperiods enables the time during which the camera module 100 resides inthe suspend state to be secured at a predetermined value, or greater, inorder for power consumption to be sufficiently reduced, even in the caseof further increasing the time during which the camera module 100resides in the suspend state in the middle of transferring image data ordecreasing the transfer period. When regularly transferring the imagedata in the static image transfer mode, the MPU 115 controls theoperations of the image sensor 103 and the ISP 105 so as to perform thecalibration only in the first transfer. In this control, the MPU 115 isable to send the parameter values acquired by calibration to the systemat the time of the first transfer of the static image data and to setthe parameter values received from the system in the second andsubsequent transfer periods.

When receiving an instruction of calibration from the system in thestatic image transfer mode, the MPU 115 performs calibration and setsnew parameter values before an immediate data transfer and sends theparameter values to the system. In addition, without sending theparameter values to the system, the MPU 115 may set the parameter valuesby reading from a flash ROM 119 previously stored parameter values, ineach transfer period. The flash ROM 119 also stores programs run by theMPU 115 and parameter values not requiring calibration. The MPU 115 isable to interpret the command received from the system to performcalibration or change the number of frames to be transferred in eachtransfer period.

The camera module 100 is a bus-powered device that operates with powersupplied from the USB bus. Note that, however, the camera module 100 maybe a self-powered device that operates with its own power. In the caseof the self-powered device, the MPU 115 controls the self-supplied powerto follow the state of the USB bus 50.

The camera module 100 transitions between a selective suspend state anda power-on state on the basis of the USB standard. The camera module 100may transition to a plurality of low-power states when it is in theselective suspend state. When operating in the static image transfermode, preferably the camera module 100 transitions to the low-powerstate in which the power consumption is the lowest when there is no needto transition to the power-on state in order to send the image data. Thepower consumption in the selective suspend state in which the powerconsumption is the lowest is preferably 10% or lower of the powerconsumption in the power-on state or the photographing state.

FIGS. 1 and 2 simplistically illustrate the main hardware configurationand the relation of connection related to the present embodiment todescribe the present embodiment. In addition to the devices mentioned inthe above description, a lot of devices are used to constitute thecamera system. These devices, however, are well-known to those skilledin the art and therefore are not mentioned in detail here. A pluralityof blocks illustrated in the diagrams may be implemented into oneintegrated circuit or device or, by contrast, one block may be dividedinto a plurality of integrated circuits or devices, which areencompassed in the scope of the present invention, as long as thoseskilled in the art can make a selection arbitrarily.

[Software Configuration of Camera System]

FIG. 3 is a functional block diagram illustrating the configuration ofsoftware loaded into the note PC 10. A dynamic image application 201 isa well-known program for acquiring dynamic image data from the cameramodule 100 to display the dynamic image data on the LCD 17 or to recordit to the HDD 15. The dynamic image application 201 may be a video phoneprogram or a dynamic image photographing program. The dynamic imageapplication 201 is able to acquire a snapshot of a static image from thedynamic image data.

The static image application 202 is a new program for acquiring imagedata from the camera module 100 in the static image transfer mode toanalyze or record the image data. The static image application 202 maybe provided with a user interface for receiving a user's instruction.Although the static image application 202 may be run while the note PC10 is supplied with power from the AC/DC adapter, the camera systemoperates with low power consumption and therefore the static imageapplication 202 is able to run while it is supplied with power from thebattery pack 27. The static image application 202 may be a program forcontrolling the power of the note PC 10 by determining the direction ofthe user's face, a program for monitoring the state of machinery, aprogram for a video phone, or a program for photographing the changestate of the environment over a long time.

A streaming service 203, which is a service program provided by an OS,sends the dynamic image data in the dynamic image transfer mode to thedynamic image application 201, sends dynamic image data for monitordisplay for use in acquiring static image data to the dynamic imageapplication 201, and passes a command issued by the dynamic imageapplication 201 to a USB camera driver 207. A static image transferservice 205, which is a new program operating in the user mode of an OS,sends static image data transferred in the static image transfer mode tothe static image application 202 and sends a command from the staticimage application 202 to the USB camera driver 207.

The USB camera driver 207 is a device driver that controls the operationof the camera module 100 and controls data transfers. The USB classdriver 209 is a device driver that performs common processing defined ina USB video class. A USB bus driver 211 controls the operation of theUSB bus connected to the USB controller.

The USB bus driver 211 causes the USB bus 50 connected to the cameramodule 100 to be in an idle state when receiving an instruction formaking the camera module 100 transition to the selective suspend statefrom the USB camera driver 207. Further, the USB bus driver 211 causesthe USB bus to transition to an active state (resume state) whenreceiving an instruction for resuming or an instruction for datatransfer. A USB host controller driver 213 controls data transfer to aUSB device and the operation of the USB host controller.

[Procedure in Static Image Transfer Mode]

The following describes a procedure in which the camera module 100transfers image data in the static image transfer mode with reference toFIGS. 4 and 5. FIG. 4 is a diagram describing the operating state of thecamera module 100 in the static image transfer mode. FIG. 4A illustratesthe entire operation, FIG. 4B illustrates the details of the firsttransfer period, and FIG. 4C illustrates the details of the second andsubsequent transfer periods. FIG. 5 is a flowchart describing aprocedure for transferring image data in the static image transfer mode.

In FIG. 4A, a certain time interval for issuing a frame transfer commandin order to transfer second and subsequent image data is referred to asa transfer period T. For example, the time between time t1 for issuing aframe transfer command used to transfer the second and subsequent imagedata and time t2 for issuing a frame transfer command used to transferthird image data and time between the time t2 and time t3 for issuing aframe transfer command used to transfer fourth image data.

In block 301, the USB bus 50 transitions to the idle state and thecamera module 100 transitions to a selective suspend state. In thestatic image transfer mode, the streaming service 203 is not performedand therefore the load on the CPU 11 is small. Various registers of thecamera module 100 that sets parameter values are cleared. At time t0,the static image application 202 sends a transfer start command to theUSB camera driver 207. The transfer start command includes the transferperiod T and the number of transfer frames of image data.

It is assumed here that the transfer period is one second and the numberof transfer frames is one frame, as an example. In this case, the framerate is one fps. The frame rate is 0.5 fps if the transfer period is twoseconds and is 2 fps if the number of transfer frames is two. The USBcamera driver 207 sends one frame transfer command generated based on atransfer start command to the USB bus driver 211 via the USB classdriver 209.

In block 303, the USB bus driver 211 resumes the USB bus 50 and sendsthe first frame transfer command to the camera module 100. Upondetecting that the USB bus 50 has resumed, the SIE 111 sends a clockpulse to the MPU 115 to activate the MPU 115. The MPU 115 starts theoperation at time two and runs the frame transfer command to wake up thecamera module 100.

In block 305, the MPU 115 sets fixed parameter values, which do not needcalibration, stored in the flash ROM 119 to various registers. The MPU115 further sets parameter values acquired by causing the image sensor103 and the ISP 105 to perform calibration in order to acquire parametervalues of exposure time, white balance, the gain of the AGC circuit, andthe like.

Moreover, the MPU 115 sends the parameter values acquired by calibrationin response to a request from the USB camera driver 207. The USB cameradriver 207 stores the received parameter values into the main memory 13.Further, the MPU 115 resets the rolling shutters for all pixels in orderand reaches time tx0. About one to two seconds are required from thetime two when the camera module 100 wakes up to the time tx0 when thephotographing preparation is completed.

In block 307, the camera module 100 starts to transfer the image data ofthe first frame by scanning the pixels in order at the time tx0 to readout image signals upon the end of the reset of the rolling shutters. TheSIE 111 packetizes the pixel data output from the encoder 107 for eachpredetermined block and outputs the packets to the static imageapplication 202. The MPU 115 controls the image sensor 103 to generateonly the image data of one frame. In block 309, the MPU 115 completesthe output of the image data of one frame at time ty0. At this timepoint, the camera module 100 may maintain the power-on state as a powerstate or may transition to the low-power state in which powerconsumption is slightly lower than the power-on state. The camera module100, however, does not transition to the selective suspend state inwhich the power consumption is the lowest.

Since the frame rate of the image sensor 103 is 30 fps, time from thetime tx0 to the time ty0 is approximately 1/30^(th) of a second (33milliseconds). In block 311, upon detecting that the transfer of thefirst image data has been completed, the USB camera driver 207 sends abus stop command for transitioning the camera module 100 to the suspendstate to the USB bus driver 211 at time t0 r. After receiving the busstop command, the USB bus driver 211 transitions the USB bus 50 to theidle state.

Upon detecting that the USB bus 50 has transitioned to the idle state attime ts0, the SIE 111 interrupts the MPU 115 at block 313 to cause thecamera module 100 to transition to the selective suspend state in whichthe power consumption is the lowest. The transition to the selectivesuspend state erases the parameter values set to the registers, andtherefore parameter values need to be set again at the next readout.

The period of time from the time t0 to the time t1 includes thecalibration time and thus is longer than the transfer period. In block315, the USB camera driver 207, which has confirmed the completion oftransferring the first image data at the time t1, sends the second frametransfer command to the camera module 100. When sending the second frametransfer command, the USB camera driver 207 sends the parameter valuesacquired from the camera module 100 in block 305 together. In order tosend the second frame transfer command, the USB bus driver 211 resumesthe USB bus 50.

Upon detecting that the USB bus 50 has resumed, the camera module 100wakes up at time tw1. Unlike the case of transferring the image data ofthe first frame, the MPU 115 does not perform calibration beforetransferring the image data of the second and subsequent frames. The MPU115 sets the fixed parameter values stored in the flash ROM 119 and theparameter values requiring calibration, which were acquired from the USBcamera driver 207, to the respective registers.

Subsequently, the MPU 115 resets the rolling shutters, generates imagedata of one frame, and completes the output at time ty1. Thephotographing preparation time after the second frame transfer commandis issued at the time t1, but before the MPU 115 is ready to generateimage data, may be about 36 milliseconds or shorter total, including thereset time for rolling shutters, assuming the parameter setting time isabout three milliseconds. Therefore, the period of time from the time t1to the time ty1 can be within a range of about 70 milliseconds or lower,with the generation time (33 milliseconds) of image data for one frameadded. In addition, the image data is transferred in units of apredetermined packet without awaiting that one frame of image data isgenerated after image data of several lines is generated, and thereforethe output time may be ignored.

Upon detecting the completion of transferring the image data of oneframe at the time ty1, the USB camera driver 207 sends a bus stopcommand to the USB bus driver 211 in order to cause the camera module100 to transition to the selective suspend state in which the powerconsumption is the lowest at time t1 r. After the USB bus 50 transitionsto the idle state, the camera module 100 transitions to the selectivesuspend state at time ts1. The period of time from the time ty1 to thetime ts1 is sufficiently short relative to the period of time from thetime t1 to the time ty1.

During the period of time from the time tw1 to the time ts1 the cameramodule resides in the photographing state in each of the second andsubsequent transfer periods. This is referred to as an actual operatingtime Ta. Similarly, the period of time from the time two, fortransitioning to the photographing state in the first transfer period,to the time ts0, for transitioning to the selective suspend state, isreferred to as an actual operating time Tb. As the rate of the actualoperating time Ta, Tb to the transfer period T is smaller, the powerconsumption of the camera module 100 is lower relative to unit imagedata.

After starting the transfer of image data, the static image application202 is able to end the transfer of static image data by issuing atransfer end command to the camera module 100 at any time. In block 316,if the camera module 100 has already received the transfer end command,the processing proceeds to block 317 to end the static image transfermode and to transition to the selective suspend state. Unless the cameramodule 100 has received the transfer end command, the processingproceeds to block 319.

In this embodiment, when the image data is transferred, the calibrationof the camera module 100 is performed only in the first transfer periodand parameter values acquired by the calibration beforehand are used inthe second and subsequent transfer periods on the assumption that thephotographing conditions do not change. If the transfer of image data inthe static image transfer mode is continued for a long time, however, aphotographing condition of illuminance changes, by which a clear imagemight not be able to be obtained.

The static image application 202 issues a rerun command for rerunningcalibration when it determines that the direction of the face cannot berecognized from the received image data or recognized image degradationby a well-known method. In block 319, if the static image application202 issued the rerun command, the processing proceeds to block 321. Inblock 321, the camera module 100 runs calibration in the same procedureas block 305 and sends acquired parameter values to the USB cameradriver 207.

When issuing a frame transfer command after receiving the parametervalues, the USB camera driver 207 sends the updated parameter values tothe camera module 100. The camera module 100 sets the updated parametervalues received from the USB camera driver 207 to registers to generatenew image data in the subsequent transfer periods. Returning to block315, the USB camera driver 207 issues a frame transfer command at acertain period on the basis of the transfer start command acquired fromthe static image application 202, and the camera module 100 sends thestatic image data to the static image application 202 in the sameprocedure in each of the third and subsequent transfer periods.

Unless the static image application 202 issues the rerun command, theprocessing proceeds to block 323. The static image application 202 isable to change the transfer period where it is found that the acquiredimage data remarkably changes or the change is flat as a result of ananalysis of the image data. In block 323, the static image application202 determines whether a change of the transfer period is necessary. Ifit is found to be necessary, the processing proceeds to block 325;otherwise, the processing returns to block 315.

In block 325, the static image application 202 sends a change commandfor changing the transfer period to the USB camera driver 207. The USBcamera driver 207 sends the subsequent frame transfer commands to thecamera module 100 at the changed transfer period. The static imageapplication 202 changes the transfer period, namely a frame rate on thebasis of its necessity, thereby enabling image data of an amount, whichis required per unit time, to be acquired in a method that requires theleast power consumption.

In the case of acquiring a static image from the camera module 100 in aconventional method, the dynamic image application 201 has acquired thestatic image from the dynamic image data, which the streaming service203 continues to send to the dynamic image application 201 until theacquisition of the static image is completed. Therefore, a large amountof image data is uselessly discarded, thereby increasing the powerconsumption of the CPU 11 that runs the streaming service 203. In thestatic image transfer mode, the camera system 10 only needs to transferimage data of one frame by performing the minimum operation. Therefore,the power consumption of the CPU 11 and that of the USB bus 50 can belargely reduced in addition to the power consumption of the cameramodule 100.

In this embodiment, the transfer of image data equivalent to one frametakes about 33 milliseconds. When the transfer period is one second, thetime during which the camera module resides in the suspend state can besecured even if a plurality of frames are transferred in each transferperiod. Therefore, an effect of reducing power consumption can beexpected until a certain number of frames. Therefore, the number offrames for image data to be transferred in each transfer period may betwo or more.

[Features of Static Image Transfer Mode]

In one aspect, the static image transfer mode is similar to the dynamicimage transfer mode in that image data is transferred in order or at acertain period. A dynamic image is a set of static images updated perunit time, and therefore both can be distinguished by the frame rate oftransferred image data.

The higher the frame rate of a dynamic image, the smoother the motionthereof. The amount of data, however, increases correspondingly. As to aframe rate, there are used 30 fps for a NTSC system, 25 fps for a PALsystem and a SECAM system, and 15 fps for a one-segment receivingservice (One Seg) for a cell phone or a mobile terminal. Accordingly,the static image transfer mode can be considered to be a method oftransferring image data at a frame rate lower than the frame ratesprescribed in the above systems and service.

The main feature of the present invention is to transition to theselective suspend state in each transfer period before transferringimage data, achieving data transfer requiring less power consumption fora use that requires a certain frame rate, but does not require a framerate as high as that of a dynamic image. The transfer period in thestatic image transfer mode is defined by the minimum limit from anexpected value of the effect of reducing power consumption and themaximum limit from the frame rate requested by the application. Inaddition, the significance of skipping the calibration in each transferperiod is to satisfy the requirements from both viewpoints to a maximumextent.

Here, it is assumed that Tb is an actual operating time from time two totime ts0 when the calibration illustrated in FIG. 4B is performed and Tais an actual operating time from time tw1 to time ts1 when thecalibration illustrated in FIG. 4C is skipped. On this assumption, Tb/Tand Ta/T represent power-saving effects per unit static image, each ofwhich will be referred to as a power reduction rate η.

The power reduction rate represents a ratio of power consumption in thestatic image transfer mode to the power consumption of the camera module100 that does not transition to the suspend state as in the conventionaltechnique. FIG. 6 is a diagram illustrating a relationship between thetransfer period T and the power reduction rate η. A line 401 representsa power reduction rate when calibration is skipped in each transferperiod and a line 403 represents a power reduction rate when calibrationis performed in each transfer period.

The inverse of the transfer period T on the horizontal axis correspondsto a frame rate. In both cases, an increase in the transfer perioddecreases the power reduction rate in inverse proportion to theincrease. Here, the decrease in the power reduction rate means animprovement in the effect of reducing the power. When the transferperiods in the lines 401 and 403 are the actual operating times Ta andTb, respectively, the power reduction rate is 1.0 and the camera module100 does not transition to the suspend state. The actual operating timesTa and Tb are each composed of the wait time between a return from theselective suspend state and the generation of image data and the timefor actually generating and outputting the image data, and the actualoperating times Ta and Tb depend on the inherent performance of thecamera module.

It is assumed here that the actual operating times Ta and Tb are givenvalues. Then, to transition to the suspend state in each transferperiod, there needs to be a theoretical lower limit for the transferperiod or a theoretical upper limit for the frame rate. For example, ifthe actual operating time Ta in the case of no calibration is approx. 70milliseconds, 100 milliseconds obtained by adding 30 milliseconds as thesuspend transition time to 70 milliseconds can be considered to be atheoretical lower limit of the transfer period. This value correspondsto 10 fps with frame rate conversion. In the case of carrying out thecalibration, the actual operating time Tb is long such as about one totwo seconds and therefore two seconds can be considered to be atheoretical lower limit of the transfer period. This value correspondsto 0.5 fps with frame rate conversion.

In the static image transfer mode, the power reduction rate is expectedto be high to some extent in comparison with a case where the cameramodule 100 continuously maintains the power-on state. Moreover, furtherimage data needs to be updated at some frequency, which is not sofrequent as for a dynamic image, in order to accomplish a purpose of theapplication. At this point, a request of an upper limit on the powerreduction rate η arises for the static image transfer mode, and thus thestatic image application 202 makes a request of the upper limit on atransfer period or the lower limit of a frame rate.

It is assumed that is the upper limit of the power reduction rate inFIG. 6. If calibration is not performed, the transfer period needs to beset to T1 or longer in order to cause the power reduction rate to be η1or lower. If the calibration is performed, the transfer period needs tobe set to T2 or longer in order to cause the power reduction rate to beη1 or lower. Assuming that Tx is the upper limit of the transfer periodT based on the request from the application, the transfer period can beset to a value between T1 and Tx if the calibration is not performed andcan be set to a value between T2 and Tx if the calibration is performed.

As apparent from FIG. 6, if the frame rate requested from the staticimage application 202 increases and thereby the transfer period Tx issmaller than T2, calibration, if performed, disables the static imagetransfer mode to be achieved. In addition, if the upper limit of thepower reduction rate η is lowered to less than η2, the transfer period Tbecomes longer than T2. Then, if the calibration is performed in thiscondition, the static image transfer mode cannot be achieved. Moreover,for a transfer period by which the static image transfer mode can beachieved even if the calibration is performed, a lower power reductionrate is achieved when the calibration is not performed.

As an example, the lower limit of the transfer period with the upperlimit of the power reduction rate assumed to be 0.1 is about one second(the upper limit of the frame rate is one fps) if the calibration is notperformed and it is 20 seconds (the upper limit of the frame rate is0.05 fps) if the calibration is performed. Moreover, the lower limit ofthe transfer period with the upper limit of the power reduction ratelessened to 0.2 is 0.5 second (the upper limit of the frame rate is twofps) if the calibration is not performed and it is 10 seconds (the upperlimit of the frame rate is 0.1 fps) if the calibration is performed.Therefore, the practical transfer period in the static image transfermode is able to be determined based on the upper limit of the powerreduction rate and the lower limit of the frame rate requested from theapplication.

Although the description has been made with an example of the USBinterface as an interface for use in connecting the camera module to thesystem, the interface may be a local interface such as IEEE 1394. Inaddition, the camera module may be connected to the note PC 10 via awired or wireless network. Moreover, description has been made, as anexample, on the control of the wakeup of the camera module 100 and thetransition to the selective suspend state in each transfer period byusing the frame transfer command and the stop command of the USB cameradriver 207. In the present invention, however, it is possible to performthe control of the power state after receiving the transfer startcommand but before receiving the transfer end command in the firmware ofthe camera module 100.

For example, when image data is transferred with a transfer periodspecified by the static image application 202, the camera module 100 isable to determine the timing of transition to the suspend state and thetiming of transition to the photographing state by itself. In thiscondition, during the suspend state, the timing of wakeup may beacquired by supplying power only to the minimum circuits for measuringthe time or by using the discharge time of electric charges accumulatedin a capacitor.

The above has described an example in which the parameter values, whichthe camera module 100 acquired by calibration, are sent once to the USBcamera driver 207. The USB camera driver 207 sends the parameter valuesto the camera every time it sends a frame transfer command. Theparameter values, however, may be stored in the flash ROM 119 and, inthe static image transfer mode, the MPU 115 may set the parameter valuesto respective registers at every wakeup.

While the present invention has been described by using a particularembodiment illustrated in the accompanying drawings, the presentinvention is not limited to the embodiment illustrated in the drawings,and naturally any conventionally known configuration may be used as longas the effect of the present invention is achieved.

DESCRIPTION OF SYMBOLS

-   -   10 Camera system    -   100 Camera module

The present invention may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the invention is, therefore, indicatedby the appended claims rather than by the foregoing description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

What is claimed is:
 1. A method of transferring image data from a cameramodule including an image sensor to a host device, comprising the stepsof: the host device issuing a plurality of ordered transfer requests ofimage data to the camera module, wherein each transfer request of theplurality of transfer requests is issued at a predetermined transferperiod of a plurality of transfer periods corresponding to the pluralityof transfer requests, a transfer period comprising an interval of timebetween a transfer request and a subsequent transfer request; the cameramodule transitioning from a low-power state to a photographing state ineach transfer period; the camera module generating image data based on apredetermined frame rate, the frame rate determining an amount of imagedata to be generated in each transfer period, wherein the inverse of theframe rate corresponds to the transfer period; the camera moduletransferring the image data to the host device in each transfer period;and the camera module transitioning to the low-power state in eachtransfer period in response to the end of the transfer.
 2. The method ofclaim 1, wherein the image data transfer request step includes issuing atransfer request at a certain transfer period.
 3. The method of claim 2,wherein the transfer period is set to a value between an actualoperating time in the photographing state in each transfer period andthe time calculated from a frame rate needed for the host device.
 4. Themethod of claim 1, wherein the image data transferred in each transferperiod is one frame of image data.
 5. The method of claim 1, whereincalibration for adaptation to photographing conditions is skipped andpreviously-acquired parameter values are set in the camera module ineach transfer period.
 6. The method of claim 5, wherein thepreviously-acquired parameter values are those acquired by the cameramodule by running the calibration in a first transfer period.
 7. Themethod of claim 5, further comprising the steps of: the host deviceevaluating the image data; the host device issuing a request to rerunthe calibration to the camera module on the basis of a result of theevaluation; and the camera module setting the parameter values acquiredby the rerun.
 8. The method of claim 1 further comprising the steps of:the host device evaluating the image data; and the host device changingan interval between the transfer requests of the image data on the basisof a result of the evaluation.
 9. A computer program product comprisinga non-transitory computer readable storage medium storing computerusable program code executable to cause a computer capable of receivingimage data from a camera module including an image sensor to implementthe functions of: causing the camera module to transition from aphotographing state to a low-power state; issuing a plurality of orderedframe transfer commands to the camera module, wherein each frametransfer command of the plurality of frame transfer commands is issuedat a predetermined transfer period of a plurality of transfer periodscorresponding to the plurality of frame transfer commands, a transferperiod comprising an interval of time between a frame transfer commandand a subsequent frame transfer command; causing the camera module totransition from the low-power state to the photographing state in eachtransfer period; receiving image data generated by the camera module inresponse to the frame transfer command, the camera module generatingimage data based on a predetermined frame rate, the frame ratedetermining an amount of image data to be generated in each transferperiod, wherein the inverse of the frame rate corresponds to thetransfer period; and causing the camera module to transition to thelow-power state in each transfer period in response to the reception ofthe image data.
 10. A camera system comprising: a camera module thatincludes an image sensor and is capable of outputting dynamic imagedata; and a host device that is connected to the camera module andissues a plurality of ordered transfer requests of image data to thecamera module, wherein each transfer request of the plurality oftransfer requests is issued at a predetermined transfer period of aplurality of transfer periods corresponding to the plurality of transferrequests, a transfer period comprising an interval of time between atransfer request and a subsequent transfer request; wherein the cameramodule transfers the image data to the host device with repeating powertransition between a photographing state and a low-power state duringeach transfer period, the image data being generated based on apredetermined frame rate, the frame rate determining an amount of imagedata to be generated in each transfer period, wherein the inverse of theframe rate corresponds to the transfer period.
 11. The camera system ofclaim 10, wherein the camera module sets parameter values acquired byperforming calibration for adaptation to a first photographing conditionat the time of transition to the photographing state in each transferperiod.
 12. The camera system of claim 10, wherein the host devicerequests the camera module to update the parameter values.
 13. Acomputer comprising: an interface that receives image data from a cameramodule including an image sensor; a transfer requesting unit that issuesa plurality of ordered transfer requests of image data to the cameramodule, wherein each transfer request of the plurality of transferrequests is issued at a predetermined transfer period of a plurality oftransfer periods corresponding to the plurality of transfer requests, atransfer period comprising an interval of time between a transferrequest and a subsequent transfer request, the image data beinggenerated based on a predetermined frame rate, the frame ratedetermining an amount of image data to be generated in each transferperiod, wherein the inverse of the frame rate corresponds to thetransfer period; a power control unit that controls the camera module totransition to a photographing state and a low-power state in eachtransfer period; and an image processing unit that receives the imagedata from the camera module.
 14. The computer of claim 13, wherein theinterface is USB.
 15. The computer of claim 13, wherein the computer isa battery-powered portable computer.
 16. The computer of claim 15,wherein the camera module is attached to a housing of the portablecomputer.
 17. The computer of claim 15, wherein the camera module issupplied with power from the computer.
 18. A camera module capable oftransferring dynamic image data to a host device, comprising: an opticalmechanism that forms an image of a subject; an image sensor thatconverts light, by which the optical mechanism formed the image, toelectric signals; an image signal processing circuit that processesoutput signals from the image sensor; an interface circuit for the hostdevice; and a processor that controls the operation of the cameramodule; wherein the processor transfers the image data to the hostdevice in response to receiving a transfer command of a plurality oftransfer commands, wherein each transfer command of the plurality oftransfer commands is issued at a predetermined transfer period of aplurality of transfer periods corresponding to the plurality of transfercommands, a transfer period comprising an interval of time between atransfer command and a subsequent transfer command, the image data beinggenerated based on a predetermined frame rate, the frame ratedetermining an amount of image data to be generated in each transferperiod, while transitioning the camera module between a photographingstate and a low-power state in each transfer period, wherein the inverseof the frame rate corresponds to the transfer period.
 19. The cameramodule of claim 18, wherein the camera module sets parameters of atleast exposure time and white balance to parameter values acquired firstwhen the camera module transitions to the photographing state.
 20. Thecamera module of claim 18, wherein power consumption in the low-powerstate is about 10% or lower than power consumption in the photographingstate.
 21. A computer program product comprising a non-transitorycomputer readable storage medium storing computer usable program codeexecutable to cause a camera module that includes an image sensor and iscapable of outputting dynamic image data to a host device to implementthe functions of: receiving a transfer command of a plurality oftransfer commands for continuously transferring image data to the hostdevice, wherein each transfer command of the plurality of transfercommands is issued at a predetermined transfer period of a plurality oftransfer periods corresponding to the plurality of transfer commands, atransfer period comprising an interval of time between a transfercommand and a subsequent transfer command; transitioning from alow-power state to a photographing state on the basis of the transfercommand; generating and outputting image data based on a predeterminedframe rate, the frame rate determining an amount of image data to begenerated in each transfer period, wherein the inverse of the frame ratecorresponds to the transfer period; transitioning to the low-power stateafter the completion of the output of the image data; and repeating thetransition to the photographing state, the output of the image data, andthe transition to the low-power state.